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ABSTRACT 


The purpose of this thesis was te present the procedure 
and techniques used to select the fourth annual recipient of 
the Rear Admiral John J. Shieffelin Award for Excellence in 
Teaching. The data and results were analyzed and compared 
with the results from the previous year. Findings indicate 
that the best teacher in each of the eleven academic depart- 
ments 1s being selected but that distribution of the eleven 
within the top twenty positions occurs randomly. 

Recommendations were made for changes in the present 
system should it continue in use and an additional 
recommendation was made for the adoption of a new method of 


selection based on the use of student questionnaires. 
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lowe LNTRODUGAAION 


In May 1969, the Superindendent of the U.S. Naval Post- 
graduate School directed that a committee be formed each 
year from among the faculty to select the recipient of an 
excellence in teaching award to be known as the Rear Admiral 
John J. Shieffelin Award. The committee was to be composed 
of a chairman, a secretary, and five members, all of whom 
would be chosen to represent a variety of backgrounds and 
curricula. In order to retain a degree of continuity within 
the committee, only four of its members would be replaced 
each year. 

The 1973 committee consisted of the following faculty 
members: 

G.J. Hokenson - Chairman 
Ne Plotkin - Secretary 


D.E. Harrison - Member 


H.J. Zweig . - Member 
JIN. Dye - Member 
Kee ovolr1 - Member 
H.J. Larson - Member 


The following guidelines for the selection process were 
adopted by the committee: 

1. All faculty members who taught two or more course 
segments in 1972 were eligible to receive the award with 
the exception of members of the selection committee, curric- 
ular and liaison officers, and previous award winners. 
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2. The voting population would consist of all current 
students, faculty, curricular officers and the Naval officer 
alumni who were graduated in 1971. 

3. Participation in the voting would be on a voluntary 
basis. 

-4, Care and discretion would be exercised to insure 
that no ranking of candidates was published. 

5. For the ballot to be valid, the voter must indicate 
on the candidate list at least five members of the faculty 
with whom he is sufficiently acquainted to make a judgment. 

6. Should the committee be unable to decide upon a 
Single dominant winner, equal multiple awards would be 
presented not to exceed three in number. 

In order to maintain complete objectivity among the 
selection committee it was decided that handling of all 
ballots and collation of all data would be performed by a 
graduate student who would then submit the results to the 
committee for its selection. The winner's name would remain 
secret until the award was presented at the June 1973 gradu- 


atleme exercisses. 





Liew. thee Baler 


A. FORMAT 

The ballot adopted by the committee for this year's 
polling was similar to those used in previous years but in- 
cluded minor modifications based on recommendations made the 
previous year. Students, faculty, and curricular officers 
each received a ballot and return envelope to which was 
attached a cover sheet which explained the purpose of the 
award and provided instructions on completing the ballot. 

In addition to the ballot and cover sheet, alumni received 
an additional cover letter from the Superintendent. 

The ballot was composed of four parts. Part A consisted 
of information useful for gathering statistics concerning 
voter characteristics. The voter was asked to identify his 
voter category, branch of service, military rank and curric- 
ular area if military or a civilian student; if a member of 
the faculty his: rank and department were to be indicated. 

Part B consisted of three blanks to be used in indicat- 
ing first, second, and third choices for the award using the 
four-digit s.dentiti1cation number specified in Fart C. 

Part C consisted of the list of all eligible faculty 
members. Members were listed in alphabetical order within 


departments and each name was preceded by a four-digit 


A sample ballot, cover letter, and instruction sheet 
are included in Appendix A. 








identification number. The number of each faculty member 
with whom the voter was academically well acquainted was to 
be circled. From this field of selected individuals, the 
voter was then to select from one to three nominees and 
place their identification numbers in the blanks provided 
ie Part Be 

Part D was an optional area in which the voter could 


provide remarks for supporting his first place nomination. 


B. BALLOT DISTRIBUTION 

Ballots were distributed to most students by placing an 
empty ballot in each student mail center box. NMSC students 
received ballots in the classroom. Alumni who were gradu- 
ated in 1971 were selected to participate in the voting and 
their names and addresses were obtained from a data tape 
maintained in we computer facility. Faculty member ballots 
were placed in faculty mail center boxes within each curric- 
ulum office. Completed ballots were placed in the return 
envelopes provided and mailed to Code 55 BAL. When received 
the ballots were forwarded to a central location in the 


Operations Analysis Department. 


<. BALLOT VALIDATION 

The validation process consisted of insuring that the 
voter's signature was on the envelope and that the identifi- 
cation numbers of at least five faculty members in Part C 
were circled. If the envelope was unsigned or less than 
five faculty members in Part C were identified, the ballot 


was considered void and eliminated from further consideration. 
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Write-in votes in Part B were not considered. If Part B 
consisted of only write-in votes the ballot was considered 
void. If Part B consisted of a combination of write-in 
votes and valid identification numbers, the write-in votes 
were ignored. If an identification number appeared more 
than once in Part B, only its first appearance was counted. 
Each valid ballot was given a four digit identification 
number for ease of handiing and future reference. 

Of the 1292 ballots returned, 335 were void, which is a 
larger percentage than occurred in 1972. The number of void 
ballots in each voter category and the reasons for disquali- 


mrecation are as follows: 


- 
i O > 
H 2 a 
(1) . — = 
2 8 28 & 
as) (ee 
<< oD) O fr, S 
Failure to identify 5 faculty members 90 165 5°15 2s 
Failure to complete Part B 1 41 1 Jae 54 
Completed Part B with write-in votes O 2 O O 2 


Received too late to be processed 2 2 O O 4 


1D TRANSFERRING THE DATA TO COMPUTER TAPE 

Upon completion of the validation process, the informa- 
tion on each valid ballot was transferred to an IBM card in 
order that it could be processed by a series of prepared 
computer programs. The first of these ee COBOL routine 


92 


called “Transfer which checked each IBM card for possible 


2 : : : P : ; 
A listing of “Transfer” is included in Appendix B. 


a 





errors. The most frequent source of error would naturally 
arise from a typographical mistake in one of the four-digit 
faculty identification numbers. These numbers were assigned 
with this fact in mind and they were specifically devised so 
that a computer routine could be used to easily check their 
accuracy. Each instructor on the ballot was assigned a 
three-digit number which formed a vector. This vector was 
then multiplied by a fixed weighting vector to form an inner 
product, the last digit of which formed the fourth digit of 
the final identification number. For example, professor 
number 629, when multiplied by the weighting vector yields: 

3 

aoe 2 OM) I 15 PPO + Oo = 37 

1 
Thus, the final identification number would be 6297, 

When an error was detected, "Transfer" identified the 
Srror and the ballot om which it occwrred in order that it 
could be corrected and resubmitted. Those ballots found to 
be error free were transferred to a magnetic tape for 
‘further puecesaina. This procedure was continued until all 


paimlots were transferred to the tape. 


LO 





A. 


rs 


STATISTICA Leah yeis 


POPULATION CHARACTERISTICS 


Table I compares voter response within each voter cate- 


gory with that response received in 1972. 


CA TEGORY 


Student 
Alumnus 
Faculty 


@urr.Oft. 


Table I. 


# ELIGIBLE # BALLOTS % RETURNED 


VOTERS 


1S 
747 
305 
17 


VODER KRroronse 


% CHANGE OVER 


RETURNED FROM ELIGIBLE PREVEOUS YEAR 


764 
407 
109 

12 


48.5 bias 
54.5 ee 
35.8 - 6.1 
WhO, #33.0 


The figures indicate that alumnus and faculty response 


percentages have decreased from the previous year while 


Student response has increased slightly and the percentage 


of curricular officers responding has increased substantial- 


Ly « 


However, 


the valid voter response figures shown in 


Table II indicate that this slight increase in the number of 


overall responses was more than negated by the considerable 


increase in void ballots received, (335 vs. 148). 


It 


appears that additional effort must be made to not only 


encourage increased voter response, but also to insure that 


ballots are filled out correctly. 


CA TEGORY 


Student 
Alumnus 
Faculty 
Gurr: Gf £. 


TabDlewil 


# ELIGIBLE # VALID 


VOTERS 


1572 
747 
305 

17 


VALID VOTER 


% 
BALLOTS FROM 


554 
314 
83 
6 


Thal 


RESPONSE 

VALID % CHANGE OVER 
ELIGIBLE PREVIOUS YEAR 
S35 32 rT 6.1 
42.1 -19.6 

Fo bs ="f.e 

Se pe + 4,0 





B. DISTRIBUTION OF Me 

After all ballots had been placed on magnetic tape, a 
second COBOL program, "Profsort™, was run which rearranged 
thesdeta fromeballot femmaitminto a statistical mecord for 
each faculty member. The program then identified the fre- 
quency distribution of M,, the number of faculty members 
known by the ‘ea voter. Table III presents a frequency 
table of M,. The mean of M- = 16.1 with a standard devia- 
t720n Of 927. An interesting fact to note is that in all 
cases where a voter identified 55 or more faculty members 
with whom he was sufficiently familiar to judge as an in- 


structor, the voter was himself a member of the faculty. 


Table III. FREQUENCY TABLE OF Me 


CELL FREQUENCY 
5-9 236 
10-14 266 
Po Oo 181 
20-24 137 
2529 80 
30-34 28 
33-50 14 
40-44 3 
45-49 5) 
50-54 2 
s2=59 1 
60-64 1 
65-69 O 
70-74 al 
a5— 19 O 
80-84 O 
So=89 O 
90-94 O 
95-99 1 
100-104 O 
105-109 O 
110-114 i 
Pio 20 2 


28 listing of “Profsort" is included in Appendix B. 
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C. DISTRIBUTION OF N, 
Once "Profsort” had established a distribution of M. and 
compiled a statistical record for each faculty member on a 
new magnetic tape, a third COBOL progran, ‘Seances was 
run which provided a frequency distribution of N,, the 
number of voters who knew the eee professor. The mean of gn, 


wre determinea to, be 53,4 with avwstandard deviation of 5097. 


Table IV is a frequency table of N 


kK? 

Table IV. FREQUENCY TABLE OF N, 
CELL FREQUENCY 
1-20 ' 28 

21-40 88 

41-60 74 

61-80 39 : 

81-100 33 
101-120 10 
121-140 10 
141-160 O 
161-180 1 
181-200 Hl 


D. DETERMINATION OF Si 


In addition to calculating N "“Scorsort'’ calculated a 


Kk? 


numerical score for each faculty member and provided the 


following statistics: 


X1k = Number of lst place votes received by professor k 
X2k = Number of 2nd place votes received by professor k 
X3k = Number of 3rd place votes received by professor k 


Scores were calculated in the following manner: 


A weighting vector W = (4,2,1) was multiplied by 





2 JIStaInGg Of 'SCOnErsort' 1s ancluded in Appendasaee 
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ee Xik 
ik Ny 


= fraction of ies place votes to teacher K from 





the population that knew him; 1=1,2,3; k=-1j2,... 


+ 
Score = (Vik Vor Var, ) i = 4 “Atle Tei Vor ae Vane 


For example, suppose a faculty member were known by 40 
students and received four first place votes, two second 
place votes and one third place vote. His score would be 


Calculated as follows: 


ae eae = 1 
Van 40 ° Ob AD ’ er 40 
4 
Score = (3 5 a) ‘ = .52500 


Table V presents the scores in descending order with the 


corresponding values of Nios Xo Xan, and X for each 


3k 


faculty member. 


E. DOMINANCE 

In calculating the numerical scores of each candidate, 
it was decided that a first place vote should be worth more 
than a second place vote which should be worth more than a 
thrid place vote. Consequently, a weighting vector i 
(4,2,1) was chosen for use and the scores were calculated 
using these figures. However, it might be possible that 
some other W than the one used could be chosen which retained 
the relationship Wj WW. but this would result in a differ- 
ent ordering of faculty member scores. 

Recall that the score of teacher K, Sh was calculated 


by multiplying the weighting vector (W) by ve the vector 


Kk? 
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Table V. 


oie 
1.98353 
1.87179 
1.65885 
1.64103 
1.60448 
1.59459 
1.58065 
1.48438 
1.42157 
1.41912 
1.35897 
1.32653 
1 sSa2510 
1.25806 
le22222 
1.20833 
LeZeO00 
io ise 1 
1412360 
Bein k. ld 
1.10638 
1.10219 
1.08000 
1.04167 


1.03226 
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102 
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39 


49 
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36 


Ge 


20 


23 


89 


45 


A7 


YS 
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48 


ou 


15. 


me 
9 
16 
7 
IZ 
38 
9 
24 
ao 
27 
34 
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10 


ES: 


10 


2k 


30 


19 


14 


22 


18 


10 


3k 


13 


13 


11 
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Table V. 


ek 
1 02778 
iWwez222 
1.00000 
0.97087 
0.93939 
0.93939 
0.93750 
0.92929 
0.91803 
0.91429 
0.91139 
0.87952 
0.87805 
0.84211 
0.82759 
0.82143 
0.80435 
0.80000 
0.79487 
0.79310 
0.78947 
0.76364 
0.75676 
0.74074 


Og 7aac9 


Mie 
36 
A5 
55 
103 
66 
33 
A8 
99 
122 
105 
79 
83 
Al 
19 
29 
56 
A6 
75 
39 
29 
95 
55 
a7 
27 


34 


16 


x 
1k 


Zz 


am. 


IL 7 


10 


10 
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Table V. 


T 
OF f2a2/ 
0.71429 
Oevizs3 
0. 703870 
0.68966 
0.68750 
0.67500 
0.66667 
O. 66000 
0.63636 
OFESi58 
0.62590 
OVOZ500 
©. 003817 
O. 60000 
0. 60000 
0.58947 
0.58537 
0.58228 
OLa1o7t 
0.57447 
Omen 5 7 
O.20710 
0.56098 


0.55882 


7 
91 
84 
73 
277 
58 
64 
40 
12 
50 
22 
57 
139 
24 
63 
50 
20 
95 
41 
79 
69 
47 
37 
134 
41 


34 


7 
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1k 


aed 
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Table V, 


Sc 
0.54839 
0.54762 
0.54286 
0.54286 
0.54167 
0.50000 
O. 50000 
O. 50000 
0.50000 
0.49057 
0.47917 
0.47458 
0.46988 
0.46875 
0.46269 
0.46154 
0, 45833 
0.43750 
0.43750 
0.43182 
0.42353 
0. 42308 
0.42105 
0.41026 


0.40278 


Ne 
31 
42 
35 
309 
48 
104 
40 
34 
28 
53 
96 
SS, 
83 
32 
67 
39 
48 
64 
48 
44 
85 
78 
19 
39 
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= ih 
0. 40000 
0.39344 
0.38554 
0.38462 
O233272 
OngSGrss 
0.38095 
OPS795 1 
0.36667 
O07. 36585 


0.36364 


0.36111 


0.35714 
0.35714 
0.35294 
0.35294 
0.34848 
0.34222 


0.34091 


0.34043 


OfSS003 


OFsSe5 5 


Os8eee3 


OS 32559 


0232653 


25 


61 


83 


26 


81 


76 


63 


29 


30 


82 


44 


36 


28 


28 


102 


34 


66 


38 


44 


47 


101 


32 


oT 


24 


49 


19 


Xx 
1k 
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SK 
0.32609 
0.32500 
0.32500 
0.32203 
0.31250 
0.30488 
0.30000 
0.29730 
0.29730 
0.29412 
0.28125 
0.28000 
0.27869 
0.27835 
0.27778 
0.27551 
0.27439 
0.26596 
0.26316 
0.25974 
0.25714 
0.25641 
0.25490 
0.25000 


0.25000 


Se 
46 
40 
40 
59 
32 
82 
10 
3 
37 
34 
96 
3) 9) 
61 
97 
72 
98 
164 
94 
38 
7 
105 
39 
a 
84 


40 


20 
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“1 
0.24390 
Of 22561 
OV2Z2222 
On2222 2 
0), 22222 
@622222 
0.22000 
0.21951 
Ov2ae75 
0.21795 
0.21429 
0.21154 
0.21053 
0.20455 
0.20690 
0.20661 
0.20588 
0.20482 


0.20000 


0.20000" 


0.20000 
0.19853 
Omi toe 
0.19444 
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41 
124 
54 
54 
36 
18 
oO) 
41 
64 
78 
28 
cE 
19 
53 
29 
Za 
34 
83 
95 
80 
40 
136 
76 
36 
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Sk 


0.19048 


O. 18267 


Onl vei 


0.17073 
0.16667 
0.16396 
0.16129 
0.16129 
0.16129 
0.15854 
OeEmS2 
0.15000 
0.14773 
0.14667 
0.14545 
0.14035 
0.13793 
0.13793 
0.13636 


O@lSss3 


Ogiess3 


0.13043 
O- 13021 
0.12903 


0.12000 


oe 
84 
AQ 
87 
Al 
12 
61 
31 
31 
31 
82 
33 
20 
88 
75 
55 
57 
29 
29 
DE 
A5 
30 
23 
192 
31 
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ar 
OL Ano > 
0.11429 
OV rElZs0 
Oeanier 
Ovamimn! t 
Cee 1 
OP) 
OF it Ie 
Oval 1. 
Ov I 1 
OPE 1a 
0.10870 
0.10417 
0.09756 
0.08889 
0.08824 
0.08333 
ONCSsS3 
0.08197 
OT OST 7 
0.07843 
0.07692 
0.07547 
0.07143 


0.07143 


Ti 
JL 
35 
80 
99 
81 
63 
45 
45 
36 
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2 
46 
48 
41 


45 
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ie 
0.06897 
0.06818 
03067 80 
OVOCS22 
OVOGs22 
0.06250 
0.06122 
0.06098 
0.06000 
O07 05662 
0.05714 
OWO5S263 
O7OS263 
©, O5a2S 
0.05000 
0.04444 
0.03704 
0.05390 
0, G3ee3 
0.05333 
O. 65630 
OF O8030 
0.0294] 
OFO2Z35 2 


0.02273 
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29 
AG 
59 
92 
46 
16 
49 
82 
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34 
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19 
19 
39 
60 
45 
54 
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or the £ractionmger iw place votes awarded to teacher K from 

the population that knew him, 17=1,2,3; k=1,2,... Therefore, 
in order to have Sms the relationship 

+ + oa ' 

WVa x F WaVon + W3V3, > WL Var t WoVopr + W3Va,1 » K #K 


must hold. Rearranging yields, 


w_(V 


V 
1 


ape 


en 


34Va. 7 Vous pee 


= ae os 
ica Yat? * WolYo, 


Now, if all values of V,, - Vi,, = 0, with at least one 
value in which the equality does not hold, then Sym Spi 
regardless of the value of W and teacher K' can be elimina- 
ted. An even stronger condition would be met if all values 
of Vin 7 Vays pao. Gis Condation is calledsabeolute 
dominance, 

For example, suppose three teachers have the following 


values of V: 


Tr V V V 
k 1k 2k 3k 
1 ~7025 woos 4114 
2 . (ok aoe ed - 4081 
3 iOS - 3862 ,50ee 


a absolutely dominates Hoge but neither qT) nou T. is 
absolutely dominated. 

Another form of dominance is called complete dominance. 
This would occur when the condition w= Tees Ww. must be 
violated in order for a particular instructor to win. In 
the above example, if all three place votes were given 
equal weights, the extreme example which still maintains the 
relationship w= w= Wo» T) would still have a higher score 


than T.- However, if a weighting vector were allowed in 
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which a third place vote is worth more than either a first 
or a second place vote, T, would have a higher score than 

qT, - Since this is the only case in which T_ can win, we say 
that a, 1s completely dominated by r and can also be elim- 
inated from the competition. 

In order to invoke these conditions of dominance and 
hence further reduce the number of possible recipients of 
the award, a FORTRAN program called Gvoninanceie was run on 
those faculty members having the top 25 scores from 
“"Scorsort"™. The results are listed in Table VI. The AD and 
CD columns list the results of the absolute dominance and 
complete dominance tests, respectively. A 1 indicates Ty, 
1s dominated and a O indicates he is not. As can be noted, 
the absolute dominance test reduced the number of possible 


recipients to 6 and the complete dominance further reduced 


mimes number to 3. 


F,. PAIRED COMPARISONS 


Since all students were not exposed to the same instruc- 
moms, it wasetelt that atymight be ofpinterest to select 
combinations of two of the three undominated instructors 
and compute their scores using only the votes of those 
voters who indicated knowing them both. In this way the 
scores could be compared to substantiate or dispute the 


relative rankings determined using the votes of the entire 


2 listing of “Demamance™. is m@eluded in Appendix se. 
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, lees 13138 
. 16000 . 20000 
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11538 
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eee! 
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.13978 
04687 
08823 
.09558 
saeco 
14285 
.22916 
.09677 
Sls 
.09722 
. 10000 
04347 
08988 
-08888 
.08510 
.10948 
. 04000 
04166 


» 12903 


AD 


CD 





6 
population. A FORTRAN program called “Comparison" was run 


and the scores were recomputed for qT, VS. Tos qT; VS. Ty: and 


It was determined, however, that T, and T, had 


in VSa I 1 A 


2 As 
moO VOLCErS 1n Common and there did not exist a Valid baci 


for comparison. 


2s listing of “Comparison” is included in Appendix B. 
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IV. HE SERBECTILON 


Upon completion of the aforementioned score computations 


and tests the following information was presented to the 


Sommittee for their consideration and selection orf a 


winner: 
diggs 


ae 


Results of the score calculations 

The number of lst, 2nd and 3rd place votes received 
by the top 25 scorers 

Results of the dominance tests for the top 25 
scorers 

Results of the paired comparisons 

Extracts of the comments sections of ballots 


selecting T a Or Team first place, 


Loe” 4 


After discussion and deliberation, the committee 


unanimously decided to recommend to the Provost that the 


award be divided equally among teachers T,, T 


Oe and T,- 


The final decision of the Provost, however, was to present 


the award to Professor M. F. Reynolds of the Department of 


Physics and Chemistry. 
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V. FURTHER ANA Sis 


In order to determine if any one academic department was 
predominant among the top 50 or bottom 50 scorers, the dis- 
tribution for each department was determined and the 
percentage. 


involved of each department was calculated. 


Tables VII and VIII give the results for 1972 and 1973. 


Metebe Vil. DISTRIBUTION OF TOP 50 SCORERS - 1972 and 1973 
LOo72 1973 

Number and % Number and % 

Dept. Of Department Of Department 
Eiteetrical Engr. 5 Geo.) 6 (15.8%) 
Aeronautics 3B (lous 4 (19.1%) 
Aviation Safety ls (253075 1 (20.0%) 
Govt. and Humanities 20 22e27) 1 (11.1%) 
Mathematics 8 (18.2%) 8 (18.6%) 
Mechanical Engr. 4 (28.6%) 4 (23.7%) 
Meteorology 3 (25207, ) 4 (28.6%) 
Oceanography 2 (14.3%) 1 ( 7.1%) 
Speen Rsch. & Admin. Sci. el es 7s 14 (17.1%) 
Physics & Chem. 6 (13.3%) 3 ( 8.8%) 
NMSC 5 (33.3%) 4 (30.8%) 
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Table VII. 

Dept. 
Electrical Engr. a 
Aeronautics 2 
Aviation Safety 1 
Govt. and Humanities O 
Mathematics 2 
Mechanical Engr. 2 
Meteorology 4 
Oceanography 5 
See. Rsch. & Admin. Sci. a7 
Physics & Chem. 7 
NMSC | 2 


DISTRIBUTION OF BORIC SOSSGOKERSa.s 


1972 
Number and % 
Of Department 


(16.3%) 
(11.1%) 


(25.0%) 


( 6.8%) 
(14.3%) 
(33.3%) 
(35.7%) 
(22.7%) 
(15.6%) 


(13.3%) 


1972 & 1973 


8 


+ 


1973 
Number and % 
Of Department 


(21.0%) 
(19.1%) 
(40.0%) 
(11.1%) 
(18.6%) 
( 5.9%) 
(21.4%) 
(21.4%) 
(17.1%) 
(14.7%) 


( 9.2%) 


It 1s interesting to note that the top 10 scorers of 


1972 represented 9 different departments and the top 10 


scorers of 1973 represented 7 different departments. This 


would appear to indicate that.the selection process was 


successfully selecting the top man in each department but 


that placement within the top 10 to 20 positions occurred on 


ae random basis. 


In order to further investigate this 


possibility, a Spearman Rank-Order Correlation (rho) test 


was performed using the rankings of the top man in each 


department in 1972 versus the rankings of the top man in 


those same departments in 1973. 
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DEA DEPARTMENT RANK DEPARTMENT RANK 


IN 1972 IN 1972 
Meteorology 1 i | 
Mech. Engr. 2 3 
Physics & Chem. 3 | Z 
EBvec. Engr. 4 4 
Avn. Safety 5 6 
Aeronautics 6 8 
Mathematics 7 7 
Ops. Rsch. and Admin. Sci. 8 il 
Govt. and Humanities 9 9 
NMSC 10 5 
Oceanography Db 10 
ae S 6S D- 
The statistic rho =1l - was computed where 
N(N“-1) 


difference in ranking of each dept. in 1972 and 1973 


N = number of departments 


6=D° = 1092 N(N2-1) = 1320 
EnO =o] .028 = .172 
Testing the significance of rho, i.e., the hypothesis 


that rho = O, the t-statistic with 9 degrees of freedom is 


Galctilated to be N-2 . 
t = thoy/ ———- = .169 
1l-rho 


which is not significant at the .05 level of significance 
and the hypothesis is not rejected. This would appear to 
indicate that the rankings of the top men in each department 


in 1972 and 1973 were randomly distributed within the first 
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twenty scores. Further substantiation of this hypothesis 
can be seen by noting the small number of voters that the 
top 20 scorers have in common. Table IX is a matrix 
depicting the number of voters in common among the top 20 
scorers in 1973. It can easily be seen that in most cases 
where teachers are in different departments, the number of 
students in common is quite small. In order to facilitate 
comparison, the following is a list of those teachers among 


the top 20 who are in the same department: 
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Table IX. NUMBER OF COMMON VOTERS AMONG TOP 20 SCORERS 1973 


0 2 0 0 5 0 0 0 Opie O sie oo 
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6 00 1002 3 1 217 2 OS Creere ecmeeeT 
9 1 1 0 0 60 011 21 5 © 1 Ol) © © CO mCmnOmEG 
2 00 0 617 2 6 S©2 Gl 187 Sez 
6 0 0 0 6 0 8 1 1 6 0 1 0 O32 0 0 =UMOEEG 
m 56 2017 8 O § 4 6 3 812 2 7 aiee lee eu 
ogee os it 2 15 0 7 8 5 3 0 (60 Curzon mE 
os Geel 1 6 1 4 7.027 6 500 929%0 13 Vie 
eo 5 5 06 8 27 On Zoe 6 Oe eee 
peecma7 «61 2 060¢63~=65 «6 70 5: 2820 5.74 
jo 2 101 8 3 5.4.5 0 © S500 114 oe 
POO. O no Ont2 0 SOM. 2 (OiOn oO Oe erO mC 
oasis 1 1 0 2 6.46 22 370 (O6N@ 5 2 OME eE 
POON Os 7 325740m0 1 0 0 OF OO) 1 O—OnomnG 
mo 2 0 5 0 18 813 9 § 25385552 Cielo: 
Cee? Oe 2 0) 1° 2d leetgliegos 2° 0 1 0 Cael 
eo 3) 0 OS (ON{S Ou 1g SOO. 1 Omomene 
7 «4 O 3. 0 3 :y9u2] 26_00sCNNOSs SOnlG 2 econ: 
mes lUdlltCOldl lO 2 68 1SmIS Sa) Jom 70 20) mo lomo 
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VI. RECOMMENDATIONS 


It is felt that the present system can be used effec- 
tively to select within each department that faculty member 
considered most proficient; however, as a means of further 
definitive selection, the present system is considered 
inadequate. The following two proposals are presented for 


conevacration: 


A. CONTINUATION OF PRESENT SELECTION METHOD 

The present method can be continued as a means of 
selecting the top scorer in each department. The cash 
award which is normally presented to the winner can be used 
to purchase appropriate engraved plaques or other memora- 
bilia which could be presented to the 11 co-winners at the 
June graduation ceremonies. 

Should the present selection method be continued, the 
following modifications are recommended: 

i. Faculty members should not be permitted to vote. 
During the course of ballot verification numerous faculty 
members who were identified by their signatures on the 
outside envelope were found to have voted for themselves. 
No accurate figures were kept in this regard, but it is 
estimated that 25% - 35% of faculty member ballots fell 
into this category. 

The average number of eligible faculty identified 


in Part C of the ballot by faculty members was 25.8. These 
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figures include three faculty members who identified 119, 
115, and 114 persons in Part C. Excluding these three fig- 
ures from the calculations reduces the average number to 
22.5. Considering the fact that in most departments faculty 
members rarely have the opportunity to witness each other at 
work in the classroom, the figure of 22.5 appears to be 
quite high. This would indicate that personal rather than 
professional familiarity is being used as a criterion for 
iPliinegtouterart C., 

2. A cover letter from the Superintendent should be 
included with all distributed ballots. During the course 
of the year, numerous questionnaires of various types are 
distributed to students in their SMC boxes by other officer 
students attempting to use the responses for course require- 
ment studies or theses. Because of the large number 
involved, many of the questionnaires find their way to the 
SMC trash containers. Considering the fact that student 
response was less than 50%, it is felt that a large number 
of the unreturned ballots suffered a similar fate. A cover 
letter from the Superintendent, identical to that accompany- 
ing each alumnus ballot, will impress on each student that 
the ballot is of a semi-official nature and a matter in 
which the Superintendent has an interest and should increase 
student participation. 

3. The headings of Parts Band € of the ballot sheumd 


be revised. Of the 335 disqualified ballots, 275 failed to 
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identify 5 faculty members in Part C. The instruction sheet 
accompanying each ballot explains the procedure to be fol- 
lowed, but no instructions for circling the identification 
numbers of at least 5 faculty members appears on the ballot 
itself. .Furtherumore, Fart B should fellow Part C amyorder 
to be ina logical sequence since Part C is performed prior 
to Part Be 
A suggested format for the headings of Parts B and C is 
as follows: 
PART B - LIST *Or ELIGIBLE FACULTY (ALL PACULTY Whee 
TAUGHT TWO OR MORE COURSE SEGMENTS IN CALENDAR 
YEAR 1973 EXCEPT CURRICULAR AND LIAISON OFFI 
CERS, MEMBERS OF THE SELE@GTIION COMMITIEE@ANE 
PRIOR WINNERS) 
ENCIRCLE THE FOUR-DIGIT IDENTIFICATION NUMBER 
OF ALL THOSE WITH WHOSE TEACHING ABILITY YOU 
ARE SUFFICIENTLY ACQUAINTED TO MAKE A JUDGMENT. 


DO NOT CONTINUE TO PART C UNLESS YOU HAVE 
. ENCIRCLED AT LEAST FIVE NUMBERS. 


PART C - NOMINATIONS (INSERT A FOUR-DIGIT FACULTY 
IDENTIFICATION NUMBER IN EACH BLANK). WRITE- 
EN VOGES ARE NOT ACCEPIABLE . 


4, The following four-digit identification numbers 
Should be changed in order that the fourth digit conform to 
the method discussed in Part II, D of this paper: 

2552 je, 25560 
5244 to 5249 
52000 tO. saZO5 


wae = tO a2 20 
S| AAAs eee FO Vim | 2/220 


5. A number of instructors who should have been 
included were inadvertently omitted from the ballot and 


conversely, the ballot contained the names of some inelgible 


39 





faculty members. A thorough check must be made to insure 
that all eligible faculty members are included. 

6. The designator for Professor under Faculty Rank 
should be PR in lieu of RP and the designator EE for 
Electrical Engineering should be included under Faculty 


Department. 


B. ADOPTION OF A NEW SELECTION SYSTEM 

In order to attempt to select the one best teacher in 
the school, it is recommended that the present system be 
replaced by one employing a new methodology. What is 
required is a system whereby faculty members can be judged 
by different populations on the basis of various teacher 
qualities or characteristics. If the scales for judgment 
were identical, all faculty members could then be compared 
on the basis of their total scores divided by the total 
number of BHelae ons: A problem inherent in this technique, 
however, is the selection of those descriptions of a teach- 
er's classroom behavior which are highly correlated to the 
student's overall concept of effective teaching. Examina- 
tion of the various forms used by schools throughout the 
nation to evaluate teacher performance indicates a lack of 
agreement on what characteristics are important.’ Conse- 
quently, work was done at the University of Washington by 


Grace M. French in an attempt to select those teacher 


i French, G.M., College Students' Concept of Effective 


Teaching Determined by an Analysis of Teacher Ratings, Ph.D. 
Miesps, Unrvercauy OrwWashington, 1957, p. ea. 
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characteristics best suited to reflect what students con- 
sider important for teacher effectiveness. A student 
questionnaire was devised which consisted of 41 items 
gathered from forms in use throughout the country and each 
student was asked to judge his instructor on a five point 
scale on each of the items listed as well as to provide an 
overall judgment of the teacher. The 41 items were regarded 
as independent variables and the overall judgment as the 
dependent variable. It was then determined what relation- 
ship exists between the system of independent variables and 
the dependent variable; i.e., the relative contribution of 
each item to the overall judgment. This was measured as the 
Procauct of Bitso where 


th item 


th 


B. = regression weight for the 1 
ro wells ses coefficient for the i item 

The top 25 items in the order of highest contribution are 
listed in Appendix C. 

It 1s recommended that a student opinion questionnaire 
be devised for use at the Naval Postgraduate School using 
the 25 items listed in Appendix C. Since each item varies 
in its Son ALON to overall judgment, a system of weights 
should be devised for each of the items in proportion to its 


relative value. A specified portion of class time at the 


end of each quarter could be allotted for completion of the 


Validity is defined in French, p. 25, as the corre- 
lation between a predictor and that which is being 
predicted. No further explanation of Deg could be found. 
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questionnaire by all students and a score compiled for each 
faculty member which would be updated each quarter. In 
addition to selection of the annual excellence in teaching 
award recipient, a form such as that proposed could be used 
in conjunction with classroom visitation appraisals by 
heads of departments as well as publication accomplishments 
to assist in making decisions regarding promotions or 


tenure. 
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APPENDIX A 


BAL bOeF 


PART A - STATISTICAL (ENCIRCLE APPROPRIATE CODE IN EACH SECTION) 


S STUDENT 
STUDENT/ ALUMNUS/ CURRICULAR OFFICER 
Military 

Branch of Service Rank 
N Navy 01 30 
M Marine Corps 02 31 
A Army 03 Se 
C Coast Guard ok 33 

L Air Force 05 
F Foreign 06 34 
X Civilian 07 35 
36 
38 
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VOTER CATEGORY 


A ALUMNUS 


Curricular Area 


Ops Analysis 

Aero Eng 

Elec & Comm Eng 
Phys Sei & Sys Tech 
(Formerly Ordnance) 
Naval Eng & Eng Sei 
Env Sei 

Mgmt & Compt Sci 
Baccalaureate 


Def Management 


Io 





FACULTY 
FACULTY 
Rank 
IR Instructor 
AT Assistant Prof. 
AC Associate Prof. 
RP Professor 


C 


CURRICULAR OFFICER 


Department 


AE 
AO 
MN 
GH 


Aeronautics 
Aviation Safety 
Bus Ad & Econ 
Govt & Humanities 
Math 

Mech Eng 
Meteorology 
Oceanography 

Ops Analysis 
Physics & Chem 


PART B - NQ4INATIONS (INSERT A FOUR DIGIT FACULTY IDENTIFICATION NUMBER IN EACH BLANK) 


FIRST CHOICE 


SECOND CHOICE 


axSSSas 


Navy Mgmt Sys 





PART C - LIST OF ELIGIBLE FACULTY (ALL FACULTY WHO HAVE TAUGHT 


EXCEPT CURRICULAR AND LIAISON OFFICERS AND MEMBERS OF THE SELECTION COMMITTEE) 


AVIATION SAFETY PROGRAMS 
1014 
1025 
5016 


5544 
1047 


Bomberger, R. B. 
Bradbury, C. M. 
Branson, J. J. 


Nielsen, J. C. 
Wible, L. C. 


DEPARTMENT OF AERONAUTICS 


1058 
2027 
5450 
1069 
1070 
1081 
S472 
5038 
1092 
5049 
1119 
1120 
1131 
1142 
1153 
5050 
1175 
1186 
5461 
1197 
1203 


potl, R. E. 
Bank, M. H. 
pelt, R. W. 
Bennett, J.A.J. 
Biblarz, 0. 
Collins, D. J. 
Evans, R. M. 
Fuhs, A. E. 
Gawain, T. H. 
Hess, R. A. 
menr, C. H. 
Layton, D. M. 
Lindsey, G. H. 
Miller, J. A. 
Netzer, D. W. 
Power, H. L. 
Redlin, M. H. 
Sehmidt, Ls. V. 
Shreeve, R. P. 
Vavra, M. H. 
Zucker, R. D. 


1481 
5061 
1519 
1531 
1542 
1564 
1586 
1597 
1625 
1647 
1658 
1669 
1670 
1681 
4035 
1692 
1708 
5083 
5449 
1753 


DEPARTMENT OF ELEC. ENGR. 


Aditer, R. W. 
Bach, R. 
Baycura, O. M. 
Bouldry, J. M. 
Breida, S. 
Chan, S. 
Cooper, P. E. 
Cotton, M. L. 
Ewing G. D. 
Geist, J. M. 
Gerba, A., Jr. 
Houston, R. K. 
Kinky D.. 5. 
Klemm, C. F.,J0r. 
Knorr,s J. 3B. 
Marmont, G. H. 
Miller, R. L. 
Myers, H. L. 
Ohlson, J. 
Panholzer, R. 
Powers, J. P. 
Powers, V. M. 
Price, 1. CG. 
Rahe, G. A. 
Rothauge, €. H. 
Sackman, G. L. 


1797 
1803 
1814 
1825 
1836 
1847 
1858 
1869 
1870 
5438 
1881 
1892 


Sheingold, A. 
ami thy. 
Spaugy, D. A. 
Stentz, D. A, 
pcrum, R. dD; 
Terman, F. W. 
Thaler, G. J. 
Titus, H. A. 
Tumner, J. B., Jr. 
Wang, C. C. 
Ward, J... R. 
Wilcox, M. L. 


DEPARTMENT OF GOVT & HUMANITIES 


iene) 
1931 
1920 
5100 
1942 
155 
5250 


Amos, J. W., Il 
Boggess, W. C. 
Bjarnason, L. L. 
Caldwell, D. E. 
Gabel, B. B. 
Gottschalk, S. 
Hurr, B. R. 

5261 Ross, G. A. 

1997 Smith, B.M.L. 
DEPARTMENT OF MATHEMATICS 
5500 Bamford, R. C. 
2134 Bender, A. P. 
2145 Bleick, W. E 
2156 Bolles, R. C. 


A3 


5144 
2167 
2189 
2190 
2206 
2217 
2228 
2239 
2240 
2251 
2262 
4o91 
2295 
2301 
2312 
2323 
2334 
5166 
5511 
2367 
2378 
2389 
5522 
2390 
2417 
at 
2428 


TWO OR MORE COURSE SEGMENTS IN CALENDAR YEAR 1972, 


Brubaker, R. H. 
Budway, J. J. 
Chewning, W. C. 
Comstock, C. 
Davis, D. Le 
Dixon, Dek. 
Estell, R. J. 
Faulkner, F, D. 
Franke, R. 
Giarratana , J. 
Gibbons, G. D 
Hanna, R. M. 
Jayachandran, T. 
Jennings, W. 
Kildall, G. A. 
Kodres, U. R. 
Kolitz, Beek. 
Kovach, L. D. 
Logen; T. J. 
Lucas, K. R. 
Marks, H. B. 
Morris, G. W. 
Moore, L. R. 
Plerce, J. f. 
Pulliam, F. M. 
Rice, B. 
Roberts, A. B. 





DEPARTMENT OF MATHEMATICS 


p33 
2439 
2h51 
2h6e2 
5188 
 2h8h 
2h95 
2501 
2512 
2523 
2534 
2545 


DEPARTMENT OF MECHANICAL ENGR. 


2552 
2567 
2578 
5566 
_ 2589 

2590 
2606 
2617 
5577 
2628 
2639 
ppp) 
5588 
2640 
2199 
2651 
5200 


- 


Russek, I. B. 
Schoenstadt, A. L. 
Shorb, A. M. 
Singer, E. A. 
Singer, S. 
Stewart, E. J. 
Stoops, G. A. 
Syms, G. .H. 
Trahan, D. H. 
Wang, P. C. C, 
Weir, M. D. 
Wilde, C. 0. 


Brock, J. E. 
Cantin, G. 
Cooper, T. E. 
Edwards, G. R. 
Garrison, G. J. 


Houlihan, T. M. 
Kelleher, M. D. 


Marto, P. J. 
McKinnon, E. A. 
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PART D - SUPPORTING REMARKS FOR YOUR PRIMARY NOMINATION 


( OPTIONAL) 











AWARD FOR EXCELLENCE IN TEACHING 


In order to augment the existing incentives for teaching of high 
quality, an Award for Excellence in Teaching has been established by the 
Superintendent. A substantial stipend was made possible through a gift 
to the Naval Postgraduate School Foundation in honor of Rear Admiral 
John J. Shieffelin, USN. The judgments of those who have participated 
in the educational processes at the Naval Postgraduate School are of 
primary importance in determining the recipients. This poll is being 
conducted to assess the collective opinion of students, faculty, staff, 
and selected alumni. After accumulating and processing the voting 
results, the Selection Committee will make its recommendations to the 
Superintendent. 


A meaningful selection of the Award recipients is heavily dependent 
upon input from a large portion of the eligible voters. Your response 
is therefore earnestly requested. Please complete the enclosed ballot 
in accordance with the instructions below. The results of the polling, 
Other than the name of the Award recipients, will be treated as 
privileged information. 


BALLOTING PROCEDURE: 


Step 1. Complete the information requested at the top (Part A) of the 
ballot. These items are for purposes of statistical analyses 
Orley: . 


Stepre. On the list of eligible faculty (Part C), encircle the four- 
digit identification number at the left of all those with whom 
you are sufficiently acquainted to make a judgment. If you have 


Cir clea ic OG Moce=nunvcrs Ol vacua lOumnOCeed LONoueD 3. 
Otherwise, please return your ballot as directed in Step 5. 


Step 3. From the subset of faculty you have indicated in Step 2, select 
from one (1) to three (3) nominees. Indicate your preferences 
in order by placing the four-digit identification numbers of 


your nominees for first, second and third choices in the appro- 


priate spaces provided in Part B. 


Step 4. You are invited to furnish a short statement in support of your 
primary nomination. Indicate those qualities which, in your 
judgment, make him an outstanding teacher. Space is provided 
for this at the end (Part D)l@er thempalilot. 


Step 5. Place your completed ballot in the enclosed return envelope, 
Sign the envelope and return. To be counted ballots must be 
received by 15 April 1973. Ballots will be separated from the 
envelopes. 
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NAVAL POSTGRADUATE SCHOOL 
MONTEREY. CALIFORNIA - 93940 IN REPLY REFER TO: 


Dear Alumnus: 


This June awards for excellence in teaching for the third 
time will be given to members of our faculty, civilian or mili- 
tary. These will include the Rear Admiral John J. Shieffelin 
Award, made possible through a grant to the Naval Postgraduate 
School Foundation. Previous winners of the award are Professor 
John N. Dyer of the Department of Physics, Professor Eugene 
C. Haderlie of the Department of Oceanography and Assistant Pro- 
fessor Ronnie L. Alberty of the Department of Meteorology. The 
recipients are determined each year through a careful poll taken 
among alumni, students and faculty. 


In recognition of the fact that some of the qualities in- 
herent in teaching excellence may not be manifest to the student 
until certain experiences have been afforded in the field, we 
again seek an input to the poll from graduates. 


Your year group has been selected as one which should have 
had time to evaluate the quality of your instruction here without 
being too far remevedmeo recall the expertence clearly 


Would you be kind enough to read the accompanying material 
and make your selections? It would be greatly appreciated. 


Sincerely, 


SON FREEMAN 
Rear Admiral, U. S. Navy 
Superintendent 


hig ee — 


Encls. 
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APPENDIX -C 


RANK-ORDERED ITEMS OF STUDENT EVALUATION 


Interprets abstract ideas and theories clearly 
Gets students interested in his subject. 

Has increased my skills in thinking. 

Has helped broaden my interests. 

Stresses important material. 

Makes good use of examples and illustrations. 
Motivated me to do my best work. 


Inspires class confidence in his knowledge 
of the subject. 


Has given me viewpoints or appreciations. 

Is clear and understandable in his explanations. 
Seems to enjoy teaching. 

Has presented many thought-provoking ideas. 

Has given me new tools for attacking problems. 


Has made the course sufficiently difficult to 
be stimulating. 


Keeps the balance between discussion and lecture 
well suited to this type of course. 


Teaches near the class level. 

Has an adequate speaking voice. 

Makes the course practical. 

Shows interest and enthusiasm in his subject. 


Controls class discussion to prevent rambling 
and confusion. 


Has maintained an atmosphere of good feeling in 
the class. 
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Gives well organized lectures. 


Encourages initiative on the part of the 
students. 


Has a good sense of humor. 


Has been well prepared for each day's lecture. 
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